Video Analysis for Universal Multimedia Messaging

Multimedia messaging is expected to become a major application for next generation mobile computing and communication devices. Heterogeneous capabilities of these devices, however, require adaptation of multimedia messages before rendering on any specific device. To achieve this in case of video messages, video analysis has to be performed to extract the structure of the video and control the message adaptation. This paper describes the universal multimedia messaging scenario and presents a short overview of existing video structuring approaches. A computationally efficient method for analyzing a video message in this scenario is introduced and a prototype for universal multimedia messaging conforming to the emerging MPEG-7 standard will be outlined. 1 Universal multimedia messaging The rapid development of mobile devices and wireless communication networks will put devices capable of multimedia into the pockets of mobile people. Messaging applications which are today text-based will allow to send and receive multimedia content anywhere, anytime, using any device. Due to the heterogeneous capabilities of the different communication devices (e.g. screen size, computing power), the changing network characteristics while the user moves (e.g., bandwidth, delay) and the different situations in which a user is involved (e.g., in a meeting, driving), multimedia content will have to be adapted to devices, networks and user preferences. This concept is called Universal Multimedia Access (UMA). Integrating UMA functionality into an asynchronous multimedia messaging system will allow a sender to compose a message without having to know the technical characteristics of the recipient’s messaging equipment. Similarly, a recipient can instruct the system how to transmit and present messages. We will call this a Universal Multimedia Messaging (UMM) system. UMM requires meta data to be stored and transmitted along with the message in order to provide the information needed for adaptation. In the UMM case, it is advantageous to use MPEG-7 [5] as a standardized meta data format. These meta data should be generated automatically as far as possible, but can also be provided by the user to emphasize or annotate parts of the message. In this paper, we will describe how UMM can be achieved in the case of asynchronous video messages. Video analysis methods are necessary to extract the meta data from the video. We will first briefly review existing methods for video structuring and abstraction, which have originally been developed for video browsing and retrieval purposes to aid a user in rapidly understanding the essence of a video message without full replays. After that, a new analysis approach for UMM will be described. Finally, we put the analysis into the context of the M3Box, which is a prototype UMM system conforming to MPEG-7.